A concept for the simultaneous orientation of brightness and range images

Within the scope of a common evaluation of brightness and range images, this article introduces a new area based approach to achieve the simultaneous orientation of both data types. The actual innovation is the combined least-squares adjustment. The adjustment is an extension of object space image matching with ranges as additional observations. The complete mathematic model is specified and discussed. For a representation of complex object surfaces, the simultaneous consideration of multiple surface patches is described. It is shown, that this concept is a generalization of the known methods for brightness and range image orientation, including approaches using area based and feature based primitives. A general overview of the orientation of brightness images is given by Heipke (1997). In the context of this paper only object based image matching algorithms are relevant. These algorithms are published in detail in the literature, see e.g. by Heipke (1990), Schneider (1991) and Weisensee (1992). The functional model includes the sensor parameters, the image orientation and the parameter of the surface function. Kempa (1995) demonstrates the estimation of the image orientation beside the surface reconstruction. One of the weaknesses of the listed approaches is the use of one single data source only. By combining brightness and range images an improvement in the evaluation can be achieved. Especially fixed relative orientations between imaging and range sensors are helpful. In the next section, we describe the potential of brightness and range images. Limitations and advantages are briefly discussed. The main part of the paper contains an introduction of a new orientation concept combining brightness and range images in a single least-squares adjustment. The functional model of brightness and range values including the image orientation and surface parameters is described. The chosen unknowns within the adjustment are discussed, and the observation equations are given. The paper concludes with some remarks on the proposed concept. The approach proposed in this paper is presented on a theoretical basis only, the paper does not address applications examples, nor does it deal with the generation of initial values for the unknowns of the non-linear adjustment. 2 SENSOR AND IMAGE DATA 2.1 Camera and brightness image Brightness images are delivered by photographic cameras and contain radiometric information of the object scene. Cameras are passive measurement systems. Light, emitted by an external source and reflected on the object surface, is received and registered by the camera sensor. For the orientation of brightness images geometric information of a surface element is needed. If this information is unknown, it has to be estimated within the orientation process. Therefore, the surface element has to be observed from at least two positions. Also sufficient structure in the radiometric signal is necessary. By including multiple images in the orientation process redundancy is produced and therefore the accuracy and reliability of the results is increased. 2.2 Laser scanner and range image Range images are delivered by airborne and terrestrial laser scanners. Each picture element represents a triple of polar coordinates. The ranges are measured by recording the travel time of pulses from the sensor to the object surface and back. Limitations of the data acquisition occur in occluded areas, and for objects with no reflection or no reflection towards the receiver. Because of the relatively high acquisition time, a laser scanner generally can not be used as a hand-held system. 2.3 Hybrid sensor Since the existence of laser scanners the combination of the scanning device with a camera sensor has been an attractive task. The goal is to provide simultaneously range data and brightness information of the object surface with identical viewing direction. The camera can either be integrated into the scanning device or mounted on top. For achieving correspondence between the sensors the eccentricity parameters have to be determined by system calibration. This kind of sensor technology is developed to obtain directly coloured point clouds. Additionally, automatic orthophoto generation is possible. Since the relation between the radiometric image and the point cloud is given, differential rectification is feasible. The relative orientation between the sensors is usually considered as constant. 3 THE NEW ORIENTATION CONCEPT In this section a new approach for the simultaneous orientation of brightness and range images is introduced. It is a general approach to orientate multiple images of multiple sensors with and without known relative orientation. The orientation concept is based on the combination of object based image matching and the exploitation of range images. The innovation is that the irradiances of the brightness image and the ranges are combined in a leastsquares adjustment. 3.1 The functional model For the model description the definition of the coordinate systems, the orientation of the individual sensors in object space, the transformation between sensor space and object space and the definition of the object surface must be introduced, see fig. 1.